The present invention relates to the manipulation and coding of video images and particularly, but not exclusively, to the interactive manipulation and predictive coding of computer graphics images for transmission over a network to one or more remote users.
A particular problem with real time image manipulation in multimedia applications such as a network system, where user commands are sent over the network from a terminal to a remote server and images modified in response to those commands are sent back over the network to the user, is the round trip loop delay.
In current MPEG systems the minimum delay in a decoder is around 80 ms (40 ms minimum buffer delay and around 40 ms decoding time) and operation at this extreme produces low quality pictures, assuming a network interface transfer rate of around 1.5 Mb/s. With higher speeds the quality may be restored or the latency may be reduced, by reducing the buffer delay, but the higher speed capability would produce penalties in terms of the cost and complexity of the equipment required. The commensurate figure for the encoder is around 60 ms, of which around 20 ms is the time to grab one field and 40 ms is the encoding time. Again, at this extreme, the quality would be poor. The delay attributable to the network back channel, passing control messages from user to server, is about 15 ms, typically. Given standard MPEG operation, with 1.5 Mb/s digital video transmission rate and accepting minimal quality presentation, the minimum round trip delay in a remote game, or any other networked Video-on-Demand (VoD) type of application, is about 160 ms.
Typical human reaction time is of the order of 80 ms, and a system (whether communicating over a network or directly connected) that responds more slowly than this will appear sluggish to a user. Whilst, in some applications, the remote user can become accustomed to the delay due to system latency and compensate for it, applications requiring rapid user reaction will inevitably suffer. As an example in terms of directly connected systems, the U.S. Federal Aviation Authority has a limit of 150 ms permitted latency in commercial flight simulators.